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Providing quantitative data with AI Mobile COLT 
to support the reflection process in language teaching 
and pre-service teacher training: a discussion 
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Abstract. Mobile COLT is a portable platform for analysis of activities in the second 
language classroom, and is based on the well-known Communicative Orientation 
of Language Teaching (COLT) scheme (Spada & Frodhlich, 1995). It has been 
developed to facilitate real-time class analysis using a Windows tablet. This paper 
first describes the COLT analysis scheme, and expounds on the functions of Mobile 
COLT, its application in classroom practices, and the development of the Artificial 
Intelligence (AI) version (AI Mobile COLT). It also briefly reports on two studies 
carried out in Japan to examine how the use of Mobile COLT can further promote 
language teaching development. Then, the paper briefly describes a collaborative 
project initiated by the authors to explore how the AI Mobile COLT system can be 
combined with an ePortfolio platform in Moodle to provide quantitative data built 


on an evidence-based framework. 
Keywords: COLT, AI Mobile COLT, pre-service teachers, language classroom 
analysis tool. 

1. Introduction 

1.1... The COLT analysis scheme 

Various observation schemes, such as the Flanders System, the Jarvis System, 


and the Stirling Project System, have been developed and tested to qualitatively 
or quantitatively assess activities in language classrooms. One of the widely 
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used schemes is COLT, introduced by Spada and Froéhlich (1995), which uses a 
standardized scale to assess features of the teaching process (Ishizuka & Yorozuya, 
2014). Since it can quantitatively display the communicative features of the class 
as to the organization (class/group/individual work), the content (meaning/form), 
the content control (teacher/text/student), the student modality (reading/writing/ 
listening/speaking), and materials (extended/minimal, native/non-native) of the 
class, COLT has demonstrated great potential as a facilitating tool for language 
teacher development. However, a key barrier to widespread use is that the manual 
coding procedures are time-consuming and complex. Thus, Ishizuka and Kibler 
(2018) have developed Mobile COLT, a portable, half-automated version of COLT 
that facilitates real-time class analysis using a Windows tablet. The two studies 
introduced in this paper below were conducted using this version of COLT. More 
recently, the team has incorporated AI into the platform, as well (AI Mobile COLT), 
which is planned to be used for the collaborative project reported in this paper (see 
Figure 1). 


Figure 1. An example of the coded English class in Japan by AI Mobile COLT 
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1.2. Using digital video recording and ePortfolio platform 


Pellerin (2011) and Pellerin, Branch-Mueller, Nicholas, and Wei (2018) studied 
ways to better document the evolution of the teaching practices of students 
enrolled in French language teaching programs in Canada by exploring the use of 
digital video recording as a tool to document the teaching practices of pre-service 
teachers. A Moodle ePortfolio platform was used to support the storage and sharing 
of digital video recordings. Access to the video on the ePortfolio platform allows 
students to engage in self-reflection, and instructors and supervisors can provide 
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feedback to the students regarding their teaching practices. However, the reflection 
and feedback available are based mainly on qualitative analysis of the videos. 
There are no quantitative data generated through the ePorfolio platform. Access 
to quantitative data would contribute to building an evidence-based framework to 
better support the development of the teaching practices during the initial training 
of language teachers. 


1.3. Using Al Mobile COLT in pre-service training 
in the context of Canadian French immersion 


The principal objective of this collaborative project is to investigate the potential 
for integration of AI Mobile COLT with a Moodle ePortfolio platform in order to 
create a standardized framework for the quantitative analysis of video collected 
during classroom teaching. The specific aim is to explore how the AI Mobile 
COLT system can enhance the analysis of the video recordings of lessons taught 
by Canadian pre-service language teachers and posted in a Moodle ePortfolio 
platform to produce standardized feedback to improve the reflective process and 
enhance supervision during training. 


2. Teacher development using Mobile COLT 


Using Mobile COLT, two studies were carried out in Japan to examine how 
this system can help promote language teaching development. Study | involved 
observation of three teachers at different school levels: elementary, junior high, 
and high school. Each teacher was visited four or five times, and their teaching 
was analyzed using Mobile COLT. The coding results were shown to the teacher 
with a graphical image, along with oral feedback. Study 2 involved observation of 
a single elementary school English teacher on two separate occasions within a two 
month interval. The same quantitative feedback process used in Study | was taken. 
After the last visits, all of the observed classes were compared, respectively, by 
their features and communicativeness in each study. 


In the COLT scheme, activities involving group work, meaning-focused content 
(management and topics), student-controlled content, discourse (extended text), 
and materials for native speakers are considered more communicative. 


In both experiments, the participant teachers tried to change their teaching styles 
every time their language classes were observed. They attempted to improve the less 
communicative aspects of their classes. The result of their efforts were sometimes 
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successful, and sometimes not. Figure 2 shows an example of the improvement of 
communicative indexes of one participant teacher during the five observations (one 


year) in Study 1. 


Figure 2. Development of one teacher in the communicative indexes 
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Table 1, for instance, shows the analysis data of the activities conducted in English. 
Between the first and second observations, group work increased by 6%, and 
student control of the content increased by 19%. On the other hand, focus-on- 
meaning activities decreased by 12%, and the use of visual materials decreased by 


18%. 


Table 1. Development of an elementary teacher (Study 2) 


COLT categories First visit Second visit 
Time Rate Time Rate 
Group 5:23 11% 8:18 17% 
Student 3:53 12% 14:56 31% 
Management/Message 2:55 57% 21:17 45% 
Extended Text/Visual 1:51 55% 17:25 37% 
L2-NS 0:00 0% 0:00 0% 


The main outcomes that emerged from the two studies conducted in Japan using 
Mobile COLT are as follows: 


128 


Providing quantitative data with Al Mobile COLT... 


¢ the data that Mobile COLT provides to teachers can affect their teaching 
styles in a short span; 


¢« Mobile COLT can suggest the points that teachers need to improve and 
facilitate self-reflection; 


¢ Mobile COLT can provide teachers with quantitative data about the 
features of their teaching styles. 


3. Discussion and future work 


The aim of the project is to integrate the AI Mobile COLT system with the 
ePortfolio platform in Moodle in the training of French language teachers in the 
western Canadian context. 


We will first explore the feasibility of the project by testing the compatibility of 
the AI Mobile system with the Moodle ePortfolio system. In particular, we will 
examine how the segmenting and coding system used by AI Mobile COLT can 
be adapted for the context of Canadian second language teaching, and, more 
specifically, in the context of the French Immersion approach for second language 
teaching and learning. 


In order to adapt the segmentation and coding system built in the AI Mobile COLT 
system, 


¢ the AI quantitative analysis system will need to be adapted for the French 
language; 


¢ direct classroom observation in the Canadian classroom will need to be 
carried out in order to better get acquainted with the enfolding language 
learning activities in an immersion approach to a second language; and 


¢ video recordings of the lessons taught by pre-service teachers during 
their practicum will need to be collected for the purpose of testing the 
quantitative analysis of the AI Mobile analysis system embedded in the 
Moodle ePortfolio. 


Unfortunately, the project has been put on hold since the end of February because 
of the COVID-19 virus. As teachers and pre-service teachers needed to transfer to 


129 


Hiroki Ishizuka and Martine Pellerin 


remote learning at the beginning of March, direct classroom observation was no 
longer possible. Moreover, since the transfer to remote teaching was challenging 
at times for teachers, as well as pre-service teachers, it was not possible to collect 
any video recording for the purpose of the project. It is the hope that the re-opening 
of schools after the Fall of 2020 will allow the project to resume its activities. 
However, with the adoption of hybrid and online teaching in Canadian schools, it 
may also be possible to gather video recordings of lessons taught completely online 
by pre-service students. 


4. Conclusions 


The integration of the AI Mobile COLT analysis in pre-service language teacher 
training has a strong potential to provide improved follow-up on the progress 
of the pre-service teachers throughout their practicum. More specifically, the 
combination of the AI Mobile COLT system with an ePortfolio platform in 
Moodle could contribute to providing quantitative data built on an evidence-based 
framework to better support the development of their teaching practices during 
initial training. 


Moreover, in the new era of COVID-19 and online and hybrid teaching, the 
integration of the AI Mobile COLT analysis with the ePortfolio platform for pre- 
service teaching programs could promote innovative ways of online supervision 
and training for a new era. 
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